stop paging a human about upstream flapping - #19

19 of the last 30 commits to this mirror changed nothing but content/.metadata.json.

Upstream serves a fallback shell for a page now and then, so its manifest entry flips between a title and an error string:

     "content/en/news/news1226.md": {
       "url": "https://api-docs.deepseek.com/news/news1226",
-      "error": "served fallback shell for ... (canonical: https://api-docs.deepseek.com/)"
+      "title": "🚀 Introducing DeepSeek-V3"

The tree comes back dirty, and the run does the full ceremony: an agent triage call, a commit, and a Bark push titled "DeepSeek docs updated" — for zero documentation changes.

A channel that cries wolf 19 times in 30 is not a channel. This is the same argument as the digest tripwire next door arriving from the other side: there, red meant nothing because it did not separate quota from defect; here a push means nothing because it does not separate a document change from upstream having a bad afternoon.

Two questions, not one

Detect changes now answers both:

The manifest still commits — it is a real record and belongs in git. It just stops being news.

The signal that was load-bearing by accident

Silencing the flapping removes the only reason anyone would notice a page that stops coming back for good. Aggregate freshness cannot see that: 273 pages keep updating while one sits frozen, pipeline green, main fresh.

So the manifest now records error_since — stamped when a page first fails, carried across consecutive failures, dropped the moment it recovers. That is what tells a flap from a death, and scripts/stale_sources.py reads it. The workflow runs it every time, including on no-change runs, because a page going dead does not announce itself with a diff.

The threshold is measured, not guessed

Replaying every manifest commit from 2026-08-02 to 09-02 for the three flappiest pages (news250120 en + zh-cn, news1226 en) gives 27 error spells of 0, 1, 3, 4 and 5 days — longest 5:

content/en/news/news250120.md   spells: 1, 0, 5, 1, 1, 4, 4, 4 days
content/zh-cn/news/news250120.md spells: 0, 1, 0, 1, 1, 0, 0, 3, 1, 1, 4 days
content/en/news/news1226.md     spells: 0, 0, 1, 1, 1, 3, 1 days

7 days would have fired zero times across that month. It stays quiet on upstream wobble and speaks only when a page has actually stopped returning.

I first read news250120 as dead since 2026-08-08 — the commit log says "fallback shell" over and over. Replaying the manifest showed it flapping ok→ERR 16 times instead, current spell 4 days. That is exactly the page this threshold has to not fire on.

Verification

• detect logic against five real tree states (clean / metadata-only / content-only / both / untracked new page) → false·false, true·false, true·true, true·true, true·true
• write_manifest — first error stamps today; continued error carries the original date rather than bumping it; recovery drops both fields; a later flap restarts the clock
• stale_sources.py against a 9-day error, a 5-day error, a healthy page, an error with no error_since, and a malformed date → reports only the 9-day one, exit 0
• workflow parses; 10 steps with the gates above

Note

On the first run after merge every currently-failing page gets error_since = that day, so the clock starts fresh. Nothing is currently near 7 days (longest live spell is 4), so the check should stay silent — that is the expected result, not a broken check.